Papers by Cicero Nogueira dos Santos
End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems (2020.emnlp-main)
Copied to clipboard
Siamak Shakeri, Cicero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Feng Nan, Zhiguo Wang, Ramesh Nallapati, Bing Xiang
| Challenge: | Existing approaches for synthetic QA data generation have limited or no success in improving the downstream Reading Comprehension task. |
| Approach: | They propose an end-to-end approach for synthetic QA data generation using a transformer-based encoder-decoder network that is trained end- to-end to generate both answers and questions. |
| Outcome: | The proposed model outperforms current state-of-the-art methods in the domain adaptation of QA models. |
Triggering Multi-Hop Reasoning for Question Answering in Language Models using Soft Prompts and Random Walks (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods that decompose multi-hop questions into single hop sub-questions are difficult to implement. |
| Approach: | They propose to use random-walks to guide pre-trained language models to map multi-hop questions to random-walked paths that lead to the answer. |
| Outcome: | The proposed methods improve on two T5 LMs. |
Answering Ambiguous Questions through Generative Evidence Fusion and Round-Trip Prediction (2021.acl-long)
Copied to clipboard
Yifan Gao, Henghui Zhu, Patrick Ng, Cicero Nogueira dos Santos, Zhiguo Wang, Feng Nan, Dejiao Zhang, Ramesh Nallapati, Andrew O. Arnold, Bing Xiang
| Challenge: | Open-domain question answering is a task to answer questions using passages with diverse topics. |
| Approach: | They propose a model that aggregates evidence from multiple passages to adaptively predict a single answer or a set of question-answer pairs for ambiguous questions. |
| Outcome: | The proposed model achieves state-of-the-art performance on AmbigQA dataset and shows competitive performance on NQ-Open and TriviaQA. |
Memory Augmented Language Models through Mixture of Word Experts (2024.naacl-long)
Copied to clipboard
| Challenge: | Increasing the parameter count of language models has been a primary driver of improved model quality, but increasing the model size also increases the cost of training and serving the model. |
| Approach: | They propose to decouple learning capacity and FLOPs by using a mixture-of-experts approach with large knowledge-rich vocabulary based routing functions. |
| Outcome: | The proposed model outperforms the T5 family of models with similar number of FLOPs on knowledge intensive tasks and similar performance to memory augmented approaches. |
Entity-level Factual Consistency of Abstractive Text Summarization (2021.eacl-main)
Copied to clipboard
Feng Nan, Ramesh Nallapati, Zhiguo Wang, Cicero Nogueira dos Santos, Henghui Zhu, Dejiao Zhang, Kathleen McKeown, Bing Xiang
| Challenge: | Existing models exhibit entity hallucination, generating names of entities that are not present in the source document. |
| Approach: | They propose to use entity-level factual consistency to improve model quality . they propose to filter the training data to reduce entity hallucination problem . |
| Outcome: | The proposed model can reduce the entity hallucination problem by filtering the training data. |
Margin-aware Unsupervised Domain Adaptation for Cross-lingual Text Labeling (2020.findings-emnlp)
Copied to clipboard
Dejiao Zhang, Ramesh Nallapati, Henghui Zhu, Feng Nan, Cicero Nogueira dos Santos, Kathleen McKeown, Bing Xiang
| Challenge: | Existing approaches to learn a model from labeled data are expensive or prohibitive. |
| Approach: | They propose an unsupervised domain adaptation algorithm that leverages labeled data in a source domain to learn a well-performing model in . they use the Margin Disparity Discrepancy algorithm to optimize the margin loss on the source domain. |
| Outcome: | The proposed approach improves on a recent theoretical work on cross-lingual document classification and NER by a large margin. |
Contrastive Fine-tuning Improves Robustness for Neural Rankers (2021.findings-acl)
Copied to clipboard
| Challenge: | Current state-of-the-art neural rankers can deteriorate when exposed to noisy inputs or applied to a new domain. |
| Approach: | They propose a contrastive loss and ranking loss method for fine-tuning rankers that combine ranking loss and rank loss to improve their robustness to query reformulations and noise perturbations. |
| Outcome: | The proposed method outperforms data augmentation for robustifying rankers on four passage ranking datasets. |
Augmented Natural Language for Generative Sequence Labeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | generative framework for joint sequence labeling and sentence-level classification is general purpose, performing well on few-shot learning, low resource, and high resource tasks. |
| Approach: | They propose a generative framework for joint sequence labeling and sentence-level classification . their framework incorporates label semantics and shares knowledge across tasks . |
| Outcome: | The proposed model performs on few-shot learning, slot labeling, and intent classification benchmarks. |
Improving Factual Consistency of Abstractive Summarization via Question Answering (2021.acl-long)
Copied to clipboard
Feng Nan, Cicero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Kathleen McKeown, Ramesh Nallapati, Dejiao Zhang, Zhiguo Wang, Andrew O. Arnold, Bing Xiang
| Challenge: | Recent studies show that about 30% of summaries generated by neural text summarization suffer from fact fabrication. |
| Approach: | They propose an automatic evaluation metric to measure factual consistency and a learning algorithm that maximizes the metric during model training. |
| Outcome: | The proposed method improves factual consistency and overall quality of summarization models. |
Fighting Offensive Language on Social Media with Unsupervised Text Style Transfer (P18-2)
Copied to clipboard
| Challenge: | Existing methods to tackle the problem of offensive language in social media are based on machine learning. |
| Approach: | They propose a method for training encoder-decoders using non-parallel data . they use a collaborative classifier, attention and the cycle consistency loss . |
| Outcome: | The proposed method outperforms state-of-the-art text style transfer systems on Twitter and Reddit . it produces reliable non-offensive transferred sentences, the authors show . |
Beyond [CLS] through Ranking by Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on generative ranking models for Information Retrieval has focused on discriminative methods that learn a similarity function to compare questions and candidates answers. |
| Approach: | They propose to use a language model to train a ranking function that model the semantic similarity of documents and queries instead of discriminative ranking functions. |
| Outcome: | The proposed approaches are as effective as state-of-the-art discriminative models for the answer selection task and show unlikelihood losses are reduced for IR. |
Generative Context Pair Selection for Multi-hop Question Answering (2021.emnlp-main)
Copied to clipboard
Dheeru Dua, Cicero Nogueira dos Santos, Patrick Ng, Ben Athiwaratkun, Bing Xiang, Matt Gardner, Sameer Singh
| Challenge: | Recent studies have shown that discriminative training results in models that exploit these underlying biases to achieve a better held-out performance, without learning the right way to reason. |
| Approach: | They propose a generative context selection model for multi-hop QA that reasons about how the given question could have been generated given a context pair and not just independent contexts. |
| Outcome: | The proposed model outperforms the state-of-the-art model on hotpotQA while being comparable to the state of the art answering performance on adversarial held-out set. |
DualTKB: A Dual Learning Bridge between Text and Knowledge Base (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for KB construction and sentence generation are lacking in the field of knowledge transfer. |
| Approach: | They propose a dual learning approach for unsupervised text to path and path to text transfers in Commonsense Knowledge Bases. |
| Outcome: | The proposed method compares favorably to existing baselines and is a viable step towards a more advanced system for automatic KB construction/expansion and reverse operation of sentence generation from KBs. |
ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference (2022.findings-acl)
Copied to clipboard
Kai Hui, Honglei Zhuang, Tao Chen, Zhen Qin, Jing Lu, Dara Bahri, Ji Ma, Jai Gupta, Cicero Nogueira dos Santos, Yi Tay, Donald Metzler
| Challenge: | State-of-the-art neural models typically encode document-query pairs using cross-attention for re-ranking. |
| Approach: | They propose to fine tune a pretrained encoder-decoder model using document to query generation. |
| Outcome: | The proposed model achieves comparable results to more expensive approaches while being 6.8X faster. |